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Abstract 


Given an image, the identification of its constituent regions and components represents of the most challenging tasks 


in image analysis and pattern recognition, commonly known as image segmentation. In the present work, we show how 


recently developed concepts and methods based on the real-valued Jaccard and coincidence methods can be applied in 





order to achieve effective image segmentation. These concepts and methods are largely related to the generalization of 


multisets to real, possibly negative multiplicities. The two aforementioned coincidence indices are applied to quantify, 





in a strict and controllable manner, the similarity between every pair of pixes, yielding a respective complex network 


representation with enhanced modular structure. A parameter controlling the relative influence of pairs of pixels with 


the same or opposite signs on the overall result allows the selection of a suitable segmentation, which is then performed 


by using standard community finding algorithms. 


‘A picture is worth what we can understand from it.’ 


LdaFC 


1 Introduction 


Image segmentation, namely the identification of the sev- 
eral regions and components of an image (e.g. |1, 2]), cor- 
responds to one of the most challenging operations in the 
areas of image analysis, computer vision, pattern recog- 
nition and artificial intelligence. 

By extending multisets (e.g. [3, 4, 5, 6, 7, 8]) to real, 
possibly negative values [9, 10, 11], it has been possible 





to derive a respectiveliy generalized Jaccard similarity in- 
dex |12, 13]. Though extensively used for binary and cat- 
egorical data, this index had not been commonly applied 
considering negative multiplicities. Given that the Jac- 
card index is not able to take into account the relative 
interiority between the compared sets, a respective exten- 
sion has been proposed [9] which also takes into account 
the interiority (or overlap [14]) index. More specifically, 
this gives rise to the coincidence index |9], corresponding 
to the product of the Jaccard and interiority indices. 

The real-valued Jaccard and coincidence indices have 
been shown to be closely related to the generalized Kro- 
necker delta function [15, 11, 16], to the point of converg- 
ing steadily to it when the numerator in the definition 
of the real-valued Jaccard index is taken to an increasing 
odd non-negative integer power [16]. 


The specific way in which the real-valued Jaccard and 
coincidence indices work, namely penalizing in a more 
effective manner values that are relatively more distinct 
one another than typically performed by product-based 
indices including the cosine similarity and Pearson corre- 
lation, the two aforementioned multiset indices pave the 
way to impressive performance in a wide range of tasks 
involving the quantification of similarities, including but 
not being limited to: template matching [17], neuronal 
pattern recognition [18], hierarchical clustering [19], and 
transforming datasets into networks with detailed inter- 
connectivity and enhanced modularity [20]. 

In the present work, we extend the application of the 
two considered multiset indices, namely the real-valued 
Jaccard and coincidence methods, to the particularly 
challenging task of image segmentation |1, 2], with im- 
pressive results. 

We start by presenting the real-valued Jaccard and co- 





incidence methods, and then illustrate their respective po- 
tential for image segmentation. 


2 The Real-Valued Jaccard and 
Coincidence Indices 


Let the following binary operations (in the sense of tak- 





ing two operands) related to multisets extended to real, 


possibly negative multiplicities [9, 10, 11, 16): 


fng= | ssmin{sjf.sqg}de (1) 
S 
fUg= | smax{ssf,sogjdz (2) 
S 
pgs J lsp — sg|/2min{ssf,sg9}dz (3) 
S 
ege J lsp + sg|/2min{ssf,sgg}dz (4) 
S 
fig = | min {s; f, sgg} dz (5) 
S 
fog =f max{ssfisogjde (6) 
S 


where f = f(z) and g = g(z) are real valued functions 
or fields, and sẹ = sign(f(z)), sy = sign(g(z)), and srg = 
S¢8 . The extension to discrete functions and fields is 
immediate. Observe that the above formulation also takes 
into account the generalization of multisets to real-valued 
functions and fields [9, 10, 11, 16|. The operations 9 and 
1 had also been considered previously, respectively in [21] 
and [22], in the context of being possibly analogous to the 
inner product in the L1 space. 

The above binary operations arise naturally when mul- 





tisets are extended to possibly negative multiplicities as 
a consequence of the several ways in which combinations 
of the sign pairwise values can be taken while calculating 
the respective operations, through functionals over the 
free variable z. In particular, Equation 1 has been veri- 
fied to correspond to the intersection between two signed 
multisets, required so that ®1 A = ®, with ® being the 
empty multiset and A being a generic multiset. 

Observe that the terms Szy, |sf + Sg|, and |s¢ — sgl, 
here called conjoint sign functions, act as selectors of the 
portions of the functions to be integrated according to 
respective sign rules. For instance, in the case of Equa- 
tion 8, only the intervals of z in which the signs of the 
two functions are opposite are considered in the respec- 
tive integration required by the respective functional. 

Given the above multiset binary operations, we can now 
define the real-valued Jaccard index |9, 10, 15, 11] between 
two functions (or multisets) f and g as: 


7 fo Sfg min {sf ¢, 599} dz = fog 
Jag) — Ía max {sff, s49} dr = füg (7) 


where S is the combined support underlying both f and 
g. 

In order to have a parameter controlling the respective 
contributions of the pairs of values with the same or op- 


posite signs, we make: 
s=s(f9)= | lss- sslmin{ssf,sog}dz (8) 


sy = sulfa) = | lss +sslminfssf,sg}dz (9) 


and then take: 


s+(f,9,a) = [a] s- — |1 — a] s4 (11) 


which yields the following a-controlled real-valued Jac- 
card index: 


Inlf.9) = 4 Js Sal fg 0) dz (12) 


g max {s ff, Sgg} dx 


Because the Jaccard index is not able to take into 
account the relative interiority between the two func- 
tions [9], it has been complemented by considering the 
interiority index (also overlap [14]). When adapted to 
real, possibly negative values, the interiority index be- 


comes: 
7 Jomin{sfp,sgg}dz _ fg 
PED =" amS) min {S,,5,} O” 
where: 
c= d 14 
p= | sif (14) 


S; = | so9(2)a: (15) 





The coincidence index can now be expressed [9, 10, 
15, 11] as corresponding to the product between the real- 
valued Jaccard and interiority indices: 


_ _Lfnallfigl 
RED Tgm {5,5} OO 
or, in expanded form: 
Cr(f,9) = 
_ fs s+(f,g,a)dz]| Js min {ss f, sgg} dz] 17) 


[Js a {sf Í; Sgg} dz] [min {SF Sg} 


3 Multiset Image Segmentation 


Though we will henceforth limit our attention to gray- 
level images, all the proposed concepts and methods can 





be immediately extended to colored, multi-spectral or 
even multi-dimensional images. 

A gray-level image (e.g. |1, 2|) can be understood as a 
matrix Alil with? =1,2,2..,JV,- and 9 = 1,2 .0.,.N;, 
therefore having dimension Ng x Ny. Each of the elements 





Figure 1: A small gray-scale image considered for illustrating the 
multiset-based segmentation method. 


of the image matrix are called pizels, their value being 
related to the respective level of luminosity. 

Figure 1 illustrates a small gray-level image of a flower. 

Before the coincidence-based methodology proposed 
in [20] can be applied as a means to transform the im- 
age into a network with enhanced modularity, it is first 
necessary to associate suitable features to each of its con- 
stituent pixels. Among the several (actually infinite) pos- 
sibilities of doing so, here we adopt the positions (or coor- 
dinates) 7 and j of the pixel, its intensity value A |7, 7] as 





well as the average intensity in a square region of radius 
1 centered at the pixel position (i.e. a 3 x 3 window). 





Therefore, each of the image pixels becomes associated 
to the following feature vector: 


fans = [i j, A li, j1, W ii, j1 (18) 





where W [i, j| corresponds to the average pixel value 
in the 3 x 3 windos. Other features can be eventually 
assigned, including those related to color, texture, etc. 
Another interesting possibility is to take some transfor- 
mation of the coordinate values 2 and j, e.g. by weighting 
them according to density, homogeneity, etc. 

In order to get a more commensurate, non-dimensional 
representation of the above features, we perform their re- 
spective statistical standardization, which consists of mak- 
ing: 


= -a (19) 


where u and o stand for the respective mean and stan- 
dard deviation [2, 23]. 
tures will have null means and unit standard deviation, 


After standardization, the fea- 


and most of the resulting values will be comprised in the 
interval |—2, 2]. 


Figure 2 depicts the image segmentation obtained by 
using the fast greedy method [24] considering a = 0.5 
and T = 0.7. 





Figure 2: The segmentation of the image in Fig. 1 by using the 
proposed methodology considering aœ = 0.5 and T = 0.7. 


The modular network obtained from the original gray- 
level image considering a = 0.5 and T = 0.7 is shown in 
Figure 3. Each of the obtained communities gives rise to 





a respective segmented region shown in different arbitrary 
colors in Figure 2. The compact cluster at the left-hand 
side corresponds to the null border of the image required 





for the calculation of the average pixel intensities within 
the window 3 x 3. 
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Figure 3: The modular network obtained for the considered image. 





Each of the communities define a respective segmented image region, 
shown in different arbitrary colors in Fig. 2. 


Figure 4 depicts the histogram of coincidence values ob- 
tained for the considered example, which define a smooth 
distribution with a peak near 0. 
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Figure 4: The segmentation of the image in Fig. 1 by using the 
proposed methodology considering a = 0.5 and T = 0.7. 


4 Concluding Remarks 





Image segmentation has represented a substantial chal- 
lenge in the areas of image processing, image analysis, 
computer vision, pattern recognition and artificial intelli- 
gence. 

In the present work, we proposed a methods for im- 





age segmentation that is based on the representation of 
datasets into networks with detailed and enhanced mod- 
ularity obtained by using the coincidence index [20]. Fea- 
tures are extracted and associated to each of the im- 
age pixels, including their position and local luminosity. 
These datasets are then transformed into respective net- 
works. A suitable value of the parameters a is then se- 








lected so as to make the communities well-defined, which 
is then followed by the application of some community 
finding methodology. 

The impressive potential of the reported methodology 
for image segmentation is illustrated respectively to a sim- 
ple gray-level image. 

Several further developments are possible, including 
but not being limited to the following possibilities: (i) 
to consider larger and diverse images; (ii) to verify the ef- 
fect of other possible features for characterization of each 
pixel including those related to color and texture; (iii) de- 
velop automated methods for selecting the a proper value 
of a, e.g. by considering entropy maximization; and (iv) 
to apply the methodology to multi-spectral and multi- 





dimensional images. Related research is being developed 





and results are expected to be communicated opportunely. 
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